Guidance for Documenting Example Applications of Analytical Tools 


1. Overview of the Application 

This section will familiarize the user with the safety issue being addressed and the 
objective of the use of the tool in the example application 

2. Input Data 

This section will describe the data ipput to the tool in the example. Points to be 
addressed in this section include: 

• Source of data analyzed 

• Data preparation 

• Any data formatting or cleansing required 

• Interface between the tool and supporting databases or other 
analytical tools 

3. Analytical Process* 

This section will describe how the tool transforms input data into output inf ormation. 
Points to be addressed in this section include: 

• Process flow diagrams (ideally) 

• Actions required by the user 

• Automated processes performed by the tool 

• Time required to run the analysis 

4. Tool’s Output* 

This section will describe the output of the tool, that is, the result of the analysis process. 
This will familiarize the user of what the resulting inf ormation will look like, how it is 
arranged, etc. 

5. Application of the Results of the Analysis 

This section will describe how the results of the analysis were or could be used, and the 
value of the information discovered. Actions taken based on the output from this 
example could be highlighted. 


Steps in the analytical process or output of the tool should include a limited number of 
sample screen shots if possible. 



Investigation Organizer, NASA Ames Research Center 


1 . Overview of the Application 

This section will familiarize the user with the safety issue being addressed 
and the objective of the use of the tool in the example application 

The InvestigationOrganizer (10) tool enables key elements of successful 
investigation through the fusion of accident investigation methodology with 
collaborative, information sharing technology,: 

• gathering and sharing disparate types of information 

• identifying the relationships between pieces of information 

• understanding the significance of such relationships 

• preserving the evidential chain 

The tool enables the first element through a Web-based application that 
can be accessed by distributed teams to store and retrieve any type of 
digital investigation material in a secure environment. The second is 
accomplished by making the relationships between information explicit 
through the use of a semantic network — a structure that literally allows an 
investigator or team to “connect -the-dots.” The third element, the 
significance of the correlated information, is established through causality 
and consistency tests using a number of different methods embedded 
within the tool, including fault trees, event sequences, and other accident 
models. And finally, the evidence gathered and structured within the tool 
can be directly, electronically archived to preserve the evidence and 
investigative reasoning. 

With investigations like SwissAir 1 1 1 consuming 4.5 years and $40 million 
dollars and future investigation likely to increase in size and complexity, 
tools that can improve the effectiveness and efficiency of investigators and 
investigative teams are not only desirable but vital. The Investigation 
Organizer fills this need and has already demonstrated its usefulness and 
significance in real, complex investigations such as those of the Columbia 
accident and the CONTOUR mishap. 


2. Input Data 

This section will describe the data input to the tool in the example. Points 
to be addressed in this section include: 

• Source of data analyzed 

• Data preparation 

• Any data formatting or cleansing required 

• Interface between the tool and supporting databases 
or other analytical tools 



Today, a wide variety of different media and different 
instruments are used to record evidence relating to mishaps 
and accidents, which may be collected and stored at remote 
locations. This evidence can include: handwritten notes, e- 
mail, text documents, taped or transcribed interviews with 
witnesses, and multi-formatted data files and images 
generated by software and/or by hardware. When a mishap 
or accident is investigated by a team that is geographically 
dispersed, these information management and coordination 
problems are particularly acute. Even more important than 
this management and coordination is the process of 
understanding the relevance of and relationships within this 
evidence — this is the heart of the investigation process itself. 

The semantic network in IO allows users to create information items of 
different types, such as people, evidence, systems, action items, or causa! 
models. It allows investigators to store important metadata with each 
information item, such as the address of a person, or the weight of a 
system. Investigators can also attach nearly any type of file to the 
information items, including text documents, images, digital audio or video 
files. Most importantly, the semantic network allows users to create 
meaningful links between these items. An example might be when an 
investigator collects a piece of evidence that supports a hypothesis (see 
Figure 1). These meaningful links allow users to explicitly share the 
reasoning as they conduct the investigation, "connecting the dots" from 
evidence, through analysis and modeling, and into findings and 
recommendations. 
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Figure 1. Semantic Network 






Information items are entered into the 10 ontology through a 
simple web form (Figure 2) which allows the investigator to 
upload a file and include any metadata they wish. The 
optional nature of the fields in the form means an 
investigator can upload information quickly and fill in the 
details later. Paper documents can either be scanned to 
electronic form using common tools or 10 can be used as an 
index for the paper documents stored in an investigator's 
office. 
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To allow even faster entry of information into 10, 
investigators can zip numerous files together to be expanded 
in 10, or an API can be used to import data from other 
applications (for example the CAFTA fault tree tool). 
Development is ongoing to allow investigators to e-mail files 
and other standard forms directly to the system for faster 
integration and communication. 

To assess and integrate the information gathered in the 
investigations, investigators link items together via standard 
link types (Figure 3). Each type of link can only apply to 
certain types of items, and when an investigator goes to 
make a link, 10’s search function makes it easy to find the 
item to link to. In addition, a built-in inference engine allows 




the system itself to help make basic links between 
information. 
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Figure 3. Linking Data 


3. Analytical Process* 

This section will describe how the tool transforms input data into output 
information. Points to be addressed in this section include: 

• Process flow diagrams (ideally) 

• Actions required by the user 

• Automated processes performed by the tool 

• Time required to run the analysis 

Causality and consistency tests can be performed on the evidence 
gathered and entered into IO using a number of different methods 
embedded within the tool, including fault trees, event sequences, and 
other accident models. As investigators determine which evidence 
supports or refutes the hypothesized causes or elements of the causal 
model, they form explicit links between them, allowing the entire 
investigation team to see the reasoning as it develops (refer back to 
Figure 3). Investigators can also view and modify event sequences based 




Figure 5. Fault Tree Viewer 


4. Tool’s Output* 

This section will describe the output of the tool, that is, the result of the 
analysis process. This will familiarize the user of what the resulting 
information will look like, how it is arranged, etc. 

The results of the analysis processes in 10 are the integration of 
information, causality and consistency checks, and the preservation of the 
evidential chain and investigative reasoning. Further, work flow and 
logistical information are managed within 10 for the investigation team. 
Finally, recommendations and corrective actions stemming from the 
investigation can be tracked and related back to the specific findings they 
address. 

5. Application of the Results of the Analysis 

This section will describe how the results of the analysis were or could be 
used, and the value of the information discovered. Actions taken based 
on the output from this example could be highlighted. 

At the end of the investigation, the investigators have preserved the 
evidence, findings and recommendations generated in the investigation. 
These can then be implemented by various aircraft manufacturers and 













in a specialized viewer within their web browser (Figure 4). Investigators 
can re-order the sequence by clicking on the X in the link to break it, and 
clicking on the handles of two boxes to link them together. Investigators 
can also add new events and conditions to the sequence directly in the 
viewer. 
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Similarly, investigators can use a fault tree viewer (Figure 5). The fault 
tree viewer also allows investigators to manage and track the progress of 
the investigation. The colors show the status of each node in the fault 
tree, and the numbers (e.g. 6/0 on fatigue) show the number of pieces of 
evidence that support/refute that particular part of the fault tree. In both 
viewers, the MV tag on the boxes allows the investigators to move them 
around to better see what is there, and the 10 button on the boxes allows 
the investigator to return to the standard view (Figure 2) at that particular 
node. Within the fault tree viewer, the user can also limit the view of the 
tree to a particular depth or a particular branch to make it easier to view, 
as some fault trees include hundreds of nodes. 








operators to prevent this type of incident from occurring again. The 
preservation of the chain of evidence within 10 allows those teams 
implementing the recommendations to trace them back to the source 
records so they can fully understand the history and extent of the 
changes. This may also allow those teams to realize that the 
recommendations should be more broadly applied. Finally, this full 
amount of information can allow safety review teams to review the 
changes made to the system and the full information behind the 
recommendations to ensure that nothing was overlooked. As more 
investigations occur, the history built up in 10 will allow safety teams to 
begin to find larger trends in the problems that are being encountered with 
a particular type of system or operator. This can then assist in improving 
system-wide safety. 



